Method and Apparatus Cross Segment Detection in a Lidar System

ABSTRACT

The present application generally relates communications and hazard avoidance within a monitored driving environment. More specifically, the application teaches a system for improved target object detection in a vehicle equipped with a laser detection and ranging LIDAR system by simultaneously transmitting multiple lasers in an array and resolving angular ambiguities using a plurality of horizontal detectors and a plurality of vertical detectors.

BACKGROUND

The present application generally relates to autonomous andsemiautonomous vehicles. More specifically, the application teaches anapparatus for improved target object detection in a vehicle equippedwith a laser detection and ranging LIDAR system.

BACKGROUND INFORMATION

The operation of modern vehicles is becoming more automated, i.e. ableto provide driving control with less and less driver intervention.Vehicle automation has been categorized into numerical levels rangingfrom zero, corresponding to no automation with full human control, tofive, corresponding to full automation with no human control. Variousautomated driver-assistance systems, such as cruise control, adaptivecruise control, and parking assistance systems correspond to lowerautomation levels, while true “driverless” vehicles correspond to higherautomation levels.

Increasingly, vehicles being equipped to determine the environmentaround them autonomously or semiautonomous using onboard sensors. Avaluable sensor for this task is LIDAR, which is a surveying technologythat measures distance by illuminating a target with a laser light.However, fixed LiDAR systems typically require a laser for each point ina field of view, thereby requiring a large number of lasers in order toachieve a dense point cloud at a distance. It would be desirable toachieve a greater density point cloud while limiting the number of lasertransmitters.

SUMMARY

Embodiments according to the present disclosure provide a number ofadvantages. For example, embodiments according to the present disclosuremay enable independent validation of autonomous vehicle control commandsto aid in diagnosis of software or hardware conditions in the primarycontrol system. Embodiments according to the present disclosure may thusbe more robust, increasing customer satisfaction.

In accordance with an aspect of the present invention, an apparatuscomprising a laser emitting device for emitting a laser in a firstdirection, a first detector having a first detection portion fordetecting a first laser intensity and a second detection portion fordetecting a second laser intensity, wherein the first detection portionand the second detection portion are oriented in a first direction, thefirst detector further operative to generate a first control signal inresponse to the detection of the first laser intensity and the secondlaser intensity, a second detector having a third detection portion fordetecting a third laser intensity and a fourth detection portion fordetecting a fourth laser intensity, wherein the third detection portionand the fourth detection portion are oriented in a second directionwherein the second direction is orthogonal to the first direction, thesecond detector further operative to generate a second control signal inresponse to the detection of the third laser intensity and the fourthlaser intensity, a first lens for distributing a first laser refectionover a subsection of the first detection portion and the seconddetection portion, a second lens for distributing a second laserrefection over a subsection of the third detection portion and thefourth detection portion, and a processor for determining a transmitterof origin in response to the first control signal and the second controlsignal.

In accordance with another aspect of the present invention, a LiDARsystem comprising a transmitter for transmitting a light pulse, a firstdetector having a first detection portion and a second detectionportion, the first detection portion and the second detection portionhaving a first orientation, a second detector having a third detectionportion and a fourth detection portion, the third detection portion andthe fourth detection portion having a second orientation, wherein thesecond orientation is orthogonal to the first orientation, and aprocessor for determining a location of the transmitter in response to afirst ratio of the light pulse detected by the first detection portionand the second detection portion and a second ratio of the light pulsedetected by the third detection portion and the fourth detectionportion.

In accordance with another aspect of the present invention, a VCSELarray having a first transmitter for transmitting a first light pulseand a second transmitter for transmitting a second light pulse, a firstdetector array having a first vertical detector for receiving a firstportion of the first light pulse and a second vertical detector forreceiving a first portion of the second light pulse, the first detectorfurther operative to generate a first control signal in response to thefirst portion of the first light pulse and the first portion of thesecond light pulse, a second detector array having a first horizontaldetector for receiving a second portion of the first light pulse and asecond horizontal detector for receiving a second portion of the secondlight pulse, the second detector further operative to generate a secondcontrol signal in response to the second portion of the first lightpulse and the second portion of the second light pulse, and a processorfor determining a location of an object in response to the first controlsignal and the second control signal.

The above advantage and other advantages and features of the presentdisclosure will be apparent from the following detailed description ofthe preferred embodiments when taken in connection with the accompanyingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The above-mentioned and other features and advantages of this invention,and the manner of attaining them, will become more apparent and theinvention will be better understood by reference to the followingdescription of embodiments of the invention taken in conjunction withthe accompanying drawings, wherein:

FIG. 1 is a schematic diagram of a communication system including anautonomously controlled vehicle, according to an embodiment.

FIG. 2 is a schematic block diagram of an automated driving system (ADS)for a vehicle, according to an embodiment.

FIG. 3 is a diagram showing an exemplary environment for implementingthe present disclosed systems and methods.

FIG. 4 is a block diagram illustrating an exemplary implementation of anapparatus for LIDAR implementation in a vehicle.

FIG. 5 is a block diagram illustrating an exemplary implementation of amethod for LIDAR implementation in a vehicle.

FIG. 6 shows an exemplary embodiments of a proposed VCSEL array forimproved LiDAR scanning.

FIG. 7 shows a flowchart of a method for improved LiDAR scanning inaccordance with an exemplary embodiment.

The exemplifications set out herein illustrate preferred embodiments ofthe invention, and such exemplifications are not to be construed aslimiting the scope of the invention in any manner.

DETAILED DESCRIPTION

The following detailed description is merely exemplary in nature and isnot intended to limit the disclosure or the application and usesthereof. Furthermore, there is no intention to be bound by any theorypresented in the preceding background or the following detaileddescription. For example, the LiDAR sensor of the present invention hasparticular application for use on a vehicle. However, as will beappreciated by those skilled in the art, the LiDAR sensor of theinvention may have other applications.

Modern vehicles sometimes include various active safety and controlsystems, such as collision avoidance systems, adaptive cruise controlsystems, lane keeping systems, lane centering systems, etc., wherevehicle technology is moving towards semi-autonomous and fullyautonomous driven vehicles. For example, collision avoidance systems areknown in the art that provide automatic vehicle control, such asbraking, if a potential or imminent collision with another vehicle orobject is detected, and also may provide a warning to allow the driverto take corrective measures to prevent the collision. Also, adaptivecruise control systems are known that employ a forward looking sensorthat provides automatic speed control and/or braking if the subjectvehicle is approaching another vehicle. The object detection sensors forthese types of systems may use any of a number of technologies, such asshort range radar, long range radar, cameras with image processing,laser or LiDAR, ultrasound, etc. The object detection sensors detectvehicles and other objects in the path of a subject vehicle, and theapplication software uses the object detection information to providewarnings or take actions as appropriate.

LiDAR sensors are sometimes employed on vehicles to detect objectsaround the vehicle and provide a range to and orientation of thoseobjects using reflections from the objects providing multiple scanpoints that combine as a point cluster range map, where a separate scanpoint is provided for every ½° or less across the field-of-view (FOV) ofthe sensor. Therefore, if a target vehicle or other object is detectedin front of the subject vehicle, there may be multiple scan points thatare returned that identify the distance of the target vehicle from thesubject vehicle. By providing a cluster of scan return points, objectshaving various and arbitrary shapes, such as trucks, trailers, bicycle,pedestrian, guard rail, etc., can be more readily detected, where thebigger and/or closer the object to the subject vehicle the more scanpoints are provided.

Most known LiDAR sensors employ a single laser and a fast rotatingmirror to produce a three-dimensional point cloud of reflections orreturns surrounding the vehicle. As the mirror rotates, the laser emitspulses of light and the sensor measures the time that it takes the lightpulse to be reflected and returned from objects in its FOV to determinethe distance of the objects, known in the art as time-of-flightcalculations. By pulsing the laser very quickly, a three-dimensionalimage of objects in the FOV of the sensor can be generated. Multiplesensors can be provided and the images therefrom can be correlated togenerate a three-dimensional image of objects surrounding the vehicle.

One disadvantage of most know LiDAR sensors is finite angular gridresolution. The LiDAR is operative to pulse the laser at discrete anglesaround the vehicle. For example, if the laser is pulsed with an angularresolution of 0.5 degrees, at 50 meters, the cross range spacing of thefield of view is approximately 0.5 meters. For a LiDAR used in anautonomous vehicle application, a target vehicle may only reflect one ortwo of the transmitted laser pulses. A few number of hits of a targetobject at a large distance may provide insufficient information ofobject boundaries. It would be desirable to estimate the surface lengthand angle orientation of each hit point and to recover additional objectinformation.

FIG. 1 schematically illustrates an operating environment that comprisesa mobile vehicle communication and control system 10 for a motor vehicle12. The communication and control system 10 for the vehicle 12 generallyincludes one or more wireless carrier systems 60, a land communicationsnetwork 62, a computer 64, a networked wireless device 57 including butnot limited to a smart phone, tablet, or wearable device such as awatch, and a remote access center 78.

The vehicle 12, shown schematically in FIG. 1, includes a propulsionsystem 13, which may in various embodiments include an internalcombustion engine, an electric machine such as a traction motor, and/ora fuel cell propulsion system. Vehicle 12 is depicted in the illustratedembodiment as a passenger car, but it should be appreciated that anyother vehicle including motorcycles, trucks, sport utility vehicles(SUVs), recreational vehicles (RVs), marine vessels, aircraft, etc., canalso be used.

The vehicle 12 also includes a transmission 14 configured to transmitpower from the propulsion system 13 to a plurality of vehicle wheels 15according to selectable speed ratios. According to various embodiments,the transmission 14 may include a step-ratio automatic transmission, acontinuously-variable transmission, or other appropriate transmission.The vehicle 12 additionally includes wheel brakes 17 configured toprovide braking torque to the vehicle wheels 15. The wheel brakes 17may, in various embodiments, include friction brakes, a regenerativebraking system such as an electric machine, and/or other appropriatebraking systems.

The vehicle 12 additionally includes a steering system 16. Whiledepicted as including a steering wheel for illustrative purposes, insome embodiments contemplated within the scope of the presentdisclosure, the steering system 16 may not include a steering wheel.

The vehicle 12 includes a wireless communications system 28 configuredto wirelessly communicate with other vehicles (“V2V”) and/orinfrastructure (“V2I”). In an exemplary embodiment, the wirelesscommunication system 28 is configured to communicate via a wirelesslocal area network (WLAN) using IEEE 802.11 standards or by usingcellular data communication. However, additional or alternatecommunication methods, such as a dedicated short-range communications(DSRC) channel, are also considered within the scope of the presentdisclosure. DSRC channels refer to one-way or two-way short-range tomedium-range wireless communication channels specifically designed forautomotive use and a corresponding set of protocols and standards.

The propulsion system 13, transmission 14, steering system 16, and wheelbrakes 17 are in communication with or under the control of at least onecontroller 22. While depicted as a single unit for illustrativepurposes, the controller 22 may additionally include one or more othercontrollers, collectively referred to as a “controller.” The controller22 may include a microprocessor such as a central processing unit (CPU)or graphics processing unit (GPU) in communication with various types ofcomputer readable storage devices or media. Computer readable storagedevices or media may include volatile and nonvolatile storage inread-only memory (ROM), random-access memory (RAM), and keep-alivememory (KAM), for example. KAM is a persistent or non-volatile memorythat may be used to store various operating variables while the CPU ispowered down. Computer-readable storage devices or media may beimplemented using any of a number of known memory devices such as PROMs(programmable read-only memory), EPROMs (electrically PROM), EEPROMs(electrically erasable PROM), flash memory, or any other electric,magnetic, optical, or combination memory devices capable of storingdata, some of which represent executable instructions, used by thecontroller 22 in controlling the vehicle.

The controller 22 includes an automated driving system (ADS) 24 forautomatically controlling various actuators in the vehicle. In anexemplary embodiment, the ADS 24 is a so-called Level Four or Level Fiveautomation system. A Level Four system indicates “high automation”,referring to the driving mode-specific performance by an automateddriving system of all aspects of the dynamic driving task, even if ahuman driver does not respond appropriately to a request to intervene. ALevel Five system indicates “full automation”, referring to thefull-time performance by an automated driving system of all aspects ofthe dynamic driving task under all roadway and environmental conditionsthat can be managed by a human driver. In an exemplary embodiment, theADS 24 is configured to control the propulsion system 13, transmission14, steering system 16, and wheel brakes 17 to control vehicleacceleration, steering, and braking, respectively, without humanintervention via a plurality of actuators 30 in response to inputs froma plurality of sensors 26, which may include GPS, RADAR, LIDAR, opticalcameras, thermal cameras, ultrasonic sensors, and/or additional sensorsas appropriate.

FIG. 1 illustrates several networked devices that can communicate withthe wireless communication system 28 of the vehicle 12. One of thenetworked devices that can communicate with the vehicle 12 via thewireless communication system 28 is the networked wireless device 57.The networked wireless device 57 can include computer processingcapability, a transceiver capable of communicating using a short-rangewireless protocol, and a visual display 59. The computer processingcapability includes a microprocessor in the form of a programmabledevice that includes one or more instructions stored in an internalmemory structure and applied to receive binary input to create binaryoutput. In some embodiments, the networked wireless device 57 includes aGPS module capable of receiving GPS satellite signals and generating GPScoordinates based on those signals. In other embodiments, the networkedwireless device 57 includes cellular communications functionality suchthat the networked wireless device 57 carries out voice and/or datacommunications over the wireless carrier system 60 using one or morecellular communications protocols, as are discussed herein. The visualdisplay 59 may also include a touch-screen graphical user interface.

The wireless carrier system 60 is preferably a cellular telephone systemthat includes a plurality of cell towers 70 (only one shown), one ormore mobile switching centers (MSCs) 72, as well as any other networkingcomponents required to connect the wireless carrier system 60 with theland communications network 62. Each cell tower 70 includes sending andreceiving antennas and a base station, with the base stations fromdifferent cell towers being connected to the MSC 72 either directly orvia intermediary equipment such as a base station controller. Thewireless carrier system 60 can implement any suitable communicationstechnology, including for example, digital technologies such as CDMA(e.g., CDMA2000), LTE (e.g., 4G LTE or 5G LTE), GSM/GPRS, or othercurrent or emerging wireless technologies. Other cell tower/basestation/MSC arrangements are possible and could be used with thewireless carrier system 60. For example, the base station and cell towercould be co-located at the same site or they could be remotely locatedfrom one another, each base station could be responsible for a singlecell tower or a single base station could service various cell towers,or various base stations could be coupled to a single MSC, to name but afew of the possible arrangements.

Apart from using the wireless carrier system 60, a second wirelesscarrier system in the form of satellite communication can be used toprovide uni-directional or bi-directional communication with the vehicle12. This can be done using one or more communication satellites 66 andan uplink transmitting station 67. Uni-directional communication caninclude, for example, satellite radio services, wherein programmingcontent (news, music, etc.) is received by the transmitting station 67,packaged for upload, and then sent to the satellite 66, which broadcaststhe programming to subscribers. Bi-directional communication caninclude, for example, satellite telephony services using the satellite66 to relay telephone communications between the vehicle 12 and thestation 67. The satellite telephony can be utilized either in additionto or in lieu of the wireless carrier system 60.

The land network 62 may be a conventional land-based telecommunicationsnetwork connected to one or more landline telephones and connects thewireless carrier system 60 to the remote access center 78. For example,the land network 62 may include a public switched telephone network(PSTN) such as that used to provide hardwired telephony, packet-switcheddata communications, and the Internet infrastructure. One or moresegments of the land network 62 could be implemented through the use ofa standard wired network, a fiber or other optical network, a cablenetwork, power lines, other wireless networks such as wireless localarea networks (WLANs), or networks providing broadband wireless access(BWA), or any combination thereof. Furthermore, the remote access center78 need not be connected via land network 62, but could include wirelesstelephony equipment so that it can communicate directly with a wirelessnetwork, such as the wireless carrier system 60.

While shown in FIG. 1 as a single device, the computer 64 may include anumber of computers accessible via a private or public network such asthe Internet. Each computer 64 can be used for one or more purposes. Inan exemplary embodiment, the computer 64 may be configured as a webserver accessible by the vehicle 12 via the wireless communicationsystem 28 and the wireless carrier 60. Other computers 64 can include,for example: a service center computer where diagnostic information andother vehicle data can be uploaded from the vehicle via the wirelesscommunication system 28 or a third party repository to or from whichvehicle data or other information is provided, whether by communicatingwith the vehicle 12, the remote access center 78, the networked wirelessdevice 57, or some combination of these. The computer 64 can maintain asearchable database and database management system that permits entry,removal, and modification of data as well as the receipt of requests tolocate data within the database. The computer 64 can also be used forproviding Internet connectivity such as DNS services or as a networkaddress server that uses DHCP or other suitable protocol to assign an IPaddress to the vehicle 12.

The remote access center 78 is designed to provide the wirelesscommunications system 28 of the vehicle 12 with a number of differentsystem functions and, according to the exemplary embodiment shown inFIG. 1, generally includes one or more switches 80, servers 82,databases 84, live advisors 86, as well as an automated voice responsesystem (VRS) 88. These various remote access center components arepreferably coupled to one another via a wired or wireless local areanetwork 90. The switch 80, which can be a private branch exchange (PBX)switch, routes incoming signals so that voice transmissions are usuallysent to either the live adviser 86 by regular phone or to the automatedvoice response system 88 using VoIP. The live advisor phone can also useVoIP as indicated by the broken line in FIG. 1. VoIP and other datacommunication through the switch 80 is implemented via a modem (notshown) connected between the switch 80 and the network 90. Datatransmissions are passed via the modem to the server 82 and/or thedatabase 84. The database 84 can store account information such assubscriber authentication information, vehicle identifiers, profilerecords, behavioral patterns, and other pertinent subscriberinformation. Data transmissions may also be conducted by wirelesssystems, such as 802.11x, GPRS, and the like. Although the illustratedembodiment has been described as it would be used in conjunction with amanned remote access center 78 using the live advisor 86, it will beappreciated that the remote access center can instead utilize the VRS 88as an automated advisor, or a combination of the VRS 88 and the liveadvisor 86 can be used.

As shown in FIG. 2, the ADS 24 includes multiple distinct controlsystems, including at least a perception system 32 for determining thepresence, location, classification, and path of detected features orobjects in the vicinity of the vehicle. The perception system 32 isconfigured to receive inputs from a variety of sensors, such as thesensors 26 illustrated in FIG. 1, and synthesize and process the sensorinputs to generate parameters used as inputs for other controlalgorithms of the ADS 24.

The perception system 32 includes a sensor fusion and preprocessingmodule 34 that processes and synthesizes sensor data 27 from the varietyof sensors 26. The sensor fusion and preprocessing module 34 performscalibration of the sensor data 27, including, but not limited to, LIDARto LIDAR calibration, camera to LIDAR calibration, LIDAR to chassiscalibration, and LIDAR beam intensity calibration. The sensor fusion andpreprocessing module 34 outputs preprocessed sensor output 35.

A classification and segmentation module 36 receives the preprocessedsensor output 35 and performs object classification, imageclassification, traffic light classification, object segmentation,ground segmentation, and object tracking processes. Objectclassification includes, but is not limited to, identifying andclassifying objects in the surrounding environment includingidentification and classification of traffic signals and signs, RADARfusion and tracking to account for the sensor's placement and field ofview (FOV), and false positive rejection via LIDAR fusion to eliminatethe many false positives that exist in an urban environment, such as,for example, manhole covers, bridges, overhead trees or light poles, andother obstacles with a high RADAR cross section but which do not affectthe ability of the vehicle to travel along its path. Additional objectclassification and tracking processes performed by the classificationand segmentation model 36 include, but are not limited to, freespacedetection and high level tracking that fuses data from RADAR tracks,LIDAR segmentation, LIDAR classification, image classification, objectshape fit models, semantic information, motion prediction, raster maps,static obstacle maps, and other sources to produce high quality objecttracks.

The classification and segmentation module 36 additionally performstraffic control device classification and traffic control device fusionwith lane association and traffic control device behavior models. Theclassification and segmentation module 36 generates an objectclassification and segmentation output 37 that includes objectidentification information.

A localization and mapping module 40 uses the object classification andsegmentation output 37 to calculate parameters including, but notlimited to, estimates of the position and orientation of vehicle 12 inboth typical and challenging driving scenarios. These challengingdriving scenarios include, but are not limited to, dynamic environmentswith many cars (e.g., dense traffic), environments with large scaleobstructions (e.g., roadwork or construction sites), hills, multi-laneroads, single lane roads, a variety of road markings and buildings orlack thereof (e.g., residential vs. business districts), and bridges andoverpasses (both above and below a current road segment of the vehicle).

The localization and mapping module 40 also incorporates new datacollected as a result of expanded map areas obtained via onboard mappingfunctions performed by the vehicle 12 during operation and mapping data“pushed” to the vehicle 12 via the wireless communication system 28. Thelocalization and mapping module 40 updates previous map data with thenew information (e.g., new lane markings, new building structures,addition or removal of constructions zones, etc.) while leavingunaffected map regions unmodified. Examples of map data that may begenerated or updated include, but are not limited to, yield linecategorization, lane boundary generation, lane connection,classification of minor and major roads, classification of left andright turns, and intersection lane creation.

In some embodiments, the localization and mapping module 40 uses SLAMtechniques to develop maps of the surrounding environment. SLAM is anacronym for Simultaneous Localization and Mapping. SLAM techniquesconstruct a map of an environment and track an object's position withinthe environment. GraphSLAM, a variant of SLAM, employs sparse matriceswhich are used to produce a graph containing observationinterdependencies.

Object position within a map is represented by a Gaussian probabilitydistribution centered around the object's predicted path. SLAM in itssimplest form utilizes three constraints: an initial locationconstraint; a relative motion constraint, which is the object's path;and a relative measurement constraint, which is one or more measurementsof an object to a landmark.

The initial motion constraint is the initial pose (e.g., position andorientation) of the vehicle, which consists of the vehicle's position intwo or three dimensional space including pitch, roll, and yaw data. Therelative motion constraint is the displaced motion of the object whichcontains a degree of flexibility to accommodate map consistency. Therelative measurement constraint includes one or more measurements fromthe object sensors to a landmark. The initial location constraint, therelative motion constraint, and the relative measurement constraint aretypically Gaussian probability distributions. Object locating methodswithin a sensor-generated map typically employ Kalman filters, variousstatistical correlation methods such as the Pearson product-momentcorrelation, and/or particle filters.

In some embodiments, once a map is built, vehicle localization isachieved in real time via a particle filter. Particle filters, unlikeBayes or Kalman filters, accommodate non-linear systems. To locate avehicle, particles are generated around an expected mean value via aGaussian probability distribution. Each particle is assigned a numericalweight representing the accuracy of the particle position to thepredicted position. Sensor data is taken into account and the particleweights are adjusted to accommodate the sensor data. The closer theproximity of the particle to the sensor adjusted position, the greaterthe numerical value of the particle weights.

As an action command occurs, each particle is updated to a new predictedposition. Sensor data is observed at the new predicted position and eachparticle is assigned a new weight representing the accuracy of theparticle position with respect to the predicted position and sensordata. The particles are re-sampled, selecting the weights that have themost numerical magnitude, thus increasing the accuracy of the predictedand sensor-corrected object position. Typically the mean, variance, andstandard deviation of the resampled data provides the new objectposition likelihood.

Particle filter processing is expressed as:

P(H _(t) |H _(t-1) ,A _(t) ,D _(t))  Equation 1

where H_(t) is the current hypothesis, which is the object position.H_(t-1) is the previous object position, A_(t) is the action, which istypically a motor command, and D_(t) is the observable data.

In some embodiments, the localization and mapping module 40 maintains anestimate of the vehicle's global position by incorporating data frommultiple sources as discussed above in an Extended Kalman Filter (EKF)framework. Kalman filters are linear filters based on Recursive BayesianFilters. Recursive Bayesian Filters, also referred to as RecursiveBayesian Estimation, essentially substitute the posterior of anestimation into the prior position to calculate a new posterior on a newestimation iteration. This effectively yields:

P(H _(t) |H _(t-1) ,D _(t))  Equation 2

where the probability of a hypothesis H_(t) is estimated by thehypothesis at the previous iteration H_(t-1) and the data D_(t) atcurrent time t.

A Kalman filter adds an action variable A_(t) where t is a timeiteration, yielding:

P(H _(t) |H _(t-1) ,A _(t) ,D _(t))  Equation 3

where the probability of a hypothesis H_(t) is based on the previoushypothesis H_(t-1), an action A_(t), and data D_(t) at current time t.

Used extensively in robotics, a Kalman filter estimates a currentposition, which is a joint probability distribution, and based on anaction command predicts a new position which is also a joint probabilitydistribution, called a state prediction. Sensor data is acquired and aseparated joint probability distribution is calculated, called a sensorprediction.

State prediction is expressed as:

X′ _(t) =AX _(t-1) +Bμ+ε _(t)  Equation 4

where X′_(t) is a new state based on the previous state AX_(t-1), Bμ andξ_(t). Constants A and B are defined by the physics of interest, μ istypically a robotic motor command, and ξ_(t) is a Gaussian state errorprediction.

Sensor prediction is expressed as:

Z′ _(t) =CX _(t)+ε_(z)  Equation 5

where Z′_(t) is the new sensor estimate, C is a function and ξ_(z) is aGaussian sensor error prediction.

A new predicted state estimate is expressed as:

X _(EST) =X′ _(t) +K(Z _(t) −Z′ _(t))  Equation 6

where the product K(Z_(t)−Z′_(t)) is referred to as the Kalman gainfactor. If the difference between the sensor prediction Z′_(t) and theactual sensor data Z_(t). (that is, Z_(t)−Z′_(t)) is reasonably close tozero, then X′_(t) is considered to be the new state estimate. IfZ_(t)−Z′_(t) is reasonably larger than zero, the K(Z_(t)−Z′_(t)) factoris added to yield a new state estimate.

As vehicle movement information is received, the EKF updates the vehicleposition estimate while also expanding the estimate covariance. Once thesensor covariance is integrated into the EKF, the localization andmapping module 40 generates a localization and mapping output 41 thatincludes the position and orientation of the vehicle 12 with respect todetected obstacles and road features.

A vehicle odometry module 46 receives data 27 from the vehicle sensors26 and generates a vehicle odometry output 47 which includes, forexample, vehicle heading, velocity, and distance information. Anabsolute positioning module 42 receives the localization and mappingoutput 41 and the vehicle odometry information 47 and generates avehicle location output 43 that is used in separate calculations asdiscussed below.

An object prediction module 38 uses the object classification andsegmentation output 37 to generate parameters including, but not limitedto, a location of a detected obstacle relative to the vehicle, apredicted path of the detected obstacle relative to the vehicle, and alocation and orientation of traffic lanes relative to the vehicle.Bayesian models may be used in some embodiments to predict driver orpedestrian intent based on semantic information, previous trajectory,and instantaneous pose, where pose is the combination of the positionand orientation of an object.

Commonly used in robotics, Bayes' Theorem, also referred to as aBayesian filter, is a form of conditional probability. Bayes' Theorem,shown below in Equation 7, sets forth the proposition that theprobability of a hypothesis H, given data D, is equal to the probabilityof a hypothesis H times the likelihood of the data D given thehypothesis H, divided by the probability of the data P(D).

$\begin{matrix}{{P\left( H \middle| D \right)} = \frac{{P(H)}{P\left( D \middle| H \right)}}{P(D)}} & {{Equation}\mspace{14mu} 7}\end{matrix}$

P(H/D) is referred to as the posterior and P(H) is referred to as theprior. Bayes' Theorem measures a probabilistic degree of belief in aproposition before (the prior) and after (the posterior) accounting forevidence embodied in the data, D. Bayes' Theorem is commonly usedrecursively when iterated. On each new iteration, the previous posteriorbecomes the prior to produce a new posterior until the iteration iscomplete. Data on the predicted path of objects (including pedestrians,surrounding vehicles, and other moving objects) is output as an objectprediction output 39 and is used in separate calculations as discussedbelow.

The ADS 24 also includes an observation module 44 and an interpretationmodule 48. The observation module 44 generates an observation output 45received by the interpretation module 48. The observation module 44 andthe interpretation module 48 allow access by the remote access center78. A live expert or advisor, e.g. the advisor 86 illustrated in FIG. 1,can optionally review the object prediction output 39 and provideadditional input and/or override automatic driving operations and assumeoperation of the vehicle if desired or required by a vehicle situation.The interpretation module 48 generates an interpreted output 49 thatincludes additional input provided by the live expert, if any.

A path planning module 50 processes and synthesizes the objectprediction output 39, the interpreted output 49, and additional routinginformation 79 received from an online database or live expert of theremote access center 78 to determine a vehicle path to be followed tomaintain the vehicle on the desired route while obeying traffic laws andavoiding any detected obstacles. The path planning module 50 employsalgorithms configured to avoid any detected obstacles in the vicinity ofthe vehicle, maintain the vehicle in a current traffic lane, andmaintain the vehicle on the desired route. The path planning module 50uses pose-graph optimization techniques, including non-linear leastsquare pose-graph optimization, to optimize the map of car vehicletrajectories in six degrees of freedom and reduce path errors. The pathplanning module 50 outputs the vehicle path information as path planningoutput 51. The path planning output 51 includes a commanded vehicle pathbased on the vehicle route, vehicle location relative to the route,location and orientation of traffic lanes, and the presence and path ofany detected obstacles.

A first control module 52 processes and synthesizes the path planningoutput 51 and the vehicle location output 43 to generate a first controloutput 53. The first control module 52 also incorporates the routinginformation 79 provided by the remote access center 78 in the case of aremote take-over mode of operation of the vehicle.

A vehicle control module 54 receives the first control output 53 as wellas velocity and heading information 47 received from vehicle odometry 46and generates vehicle control output 55. The vehicle control output 55includes a set of actuator commands to achieve the commanded path fromthe vehicle control module 54, including, but not limited to, a steeringcommand, a shift command, a throttle command, and a brake command.

The vehicle control output 55 is communicated to actuators 30. In anexemplary embodiment, the actuators 30 include a steering control, ashifter control, a throttle control, and a brake control. The steeringcontrol may, for example, control a steering system 16 as illustrated inFIG. 1. The shifter control may, for example, control a transmission 14as illustrated in FIG. 1. The throttle control may, for example, controla propulsion system 13 as illustrated in FIG. 1. The brake control may,for example, control wheel brakes 17 as illustrated in FIG. 1.

It should be understood that the disclosed methods can be used with anynumber of different systems and is not specifically limited to theoperating environment shown here. The architecture, construction, setup,and operation of the system 10 and its individual components isgenerally known. Other systems not shown here could employ the disclosedmethods as well.

Turning now to FIG. 3, an exemplary environment 300 for implementing thepresent disclosed systems and methods is shown. In the illustrativeexample, a vehicle 310 is traveling with an operational LIDAR system.The system has a transmitter which is operative to transmit pulsed lightor lasers 330 away from the vehicle 310. Some of the pulsed light isincident on objects 320 around the vehicle and a reflected signal isreturned to a receiver on the vehicle. The vehicle is also equipped witha processor to process the returned signal to measure amplitude,propagation time and phase shift among other characteristics, in orderto determine the distance to the objects 320, as well as size andvelocity of the objects 320.

Turning now to FIG. 4, a functional block diagram of a LIDAR system 400according to an exemplary method and system is shown. LIDAR transceiver410 is operative to generate a laser beam, transmit the laser beam andcapture the laser energy scattered/reflected from an object within theFOV. Scanner 420 moves laser beam across the target areas, PositionOrientation System (POS) measures sensor position and orientation 430,system processor 440 controls all above actions, vehicle control systemand user interface 450, data storage 460.

The LIDAR transceiver 410 is operative to generate a laser beam,transmit the laser beam into the FOV and capture energy reflected from atarget. LIDAR sensors employ time-of-flight to determine the distance ofobjects from which the pulsed laser beams are reflected. The oscillatinglight signal is reflected off of the object and is detected by thedetector within the LIDAR transceiver 410 with a phase shift thatdepends on the distance that the object is from the sensor. Anelectronic phase lock loop (PLL) may be used to extract the phase shiftfrom the signal and that phase shift is translated to a distance byknown techniques. The detector may also employ peak detection.

The scanner 420 is used to move the laser beam across the FOV. In oneexemplary application, a rotational mirror is used to reflect astationary laser across the FOV. In another exemplary application, anumber of fixed lasers are pulsed in different directions in order togenerate a FOV object model.

A POS 430 is used to accurately determine the time, position andorientation of the scanner 420 when a laser is pulsed. The system mayinclude a GPS sensor, inertial measurement system, and other sensors.The POS may further be operative to determine the range measurement,scan angle, sensor position, sensor orientation and signal amplitude.The data generated by the POS 430 may be combined with data generated bythe LIDAR transceiver 410 in order generate a FOV object model.

The system processor 440 is operative to transmit control signals to theLiDAR transceiver 410, the POS 430 and the scanner 420 and to receivedata from these devices. The system processor 240 receives the data anddetermines the location of objects within the FOV, and may determineother information such as velocity of objects, composition of objects,signal filtering, etc. The memory 460 is operative to store digitalrepresentations of returned signal pulses, and/or to store datacalculated by the system processor 440. The vehicle control system/userinterface 450 is operative to receive inputs from a user, to displayresults if required, and optionally, to generate vehicle control signalsin response to the data generated by the system processor 440. Vehiclecontrol signals may be used to control an autonomous vehicle, may beused for collision avoidance, or may be used for a driver warningsystem, among other uses.

FIG. 5 is a block diagram of an array LiDAR system 500 according toembodiments. An array LiDAR 510 includes an array of lasers 511. Eachlaser 511 may be a vertical-cavity surface-emitting laser (VCSEL). AVCSEL is a semiconductor-based laser diode that emits an optical beamvertically from its top surface, as shown. The laser beam 512 emitted byeach laser 511 (e.g., VCSEL) forms a field-of-view. Any object 515within the field of view of the array LiDAR 510 result in reflections516 that are received at a bandpass filter (BPF) 520. The reflections516 in the field of view of the receive side of the array LiDAR system500 are filtered by the BPF 520 and focused through a lens 525 to anavalanche photodiode (APD) 535 that converts the received and filteredlight into an electrical signal. This electrical signal is provided to aprocessing system 530. The processing system 530 may generate the signalultimately emitted as laser beams 512. The generated signal may passthrough an amplitude modulator 505 before reaching the array LiDAR 510.

The exemplary LiDAR system 500 is operative to transmit light pulses ata known pulse duration and a known pulse repetition rate and to receivelight pulses resulting from the transmitted pulse reflection from anobject with the path of the transmitted laser pulse. A typical pulseduration may be 10 nsec where a typical pulse repetition rate may be 140kHz. A longer pulse duration may result in a lower SNR

The transmitter 510 is operative to transmit a sequence of laser pulsesin a known direction. The transmitter may include a laser diode forgenerating a laser, a wavelength modulator, a pulse width modulator, afrequency modulator, and/or an amplifier. The transmitter is operativeto scan the laser pulses over a desired field of view over a period oftime.

Receiver 530 is operative to receive the pulsed laser signals after theyare reflected from objects within the FOV. The receiver may includeamplifiers, mixers, circulators and the like in order to convert thereceived pulsed laser signal into an intermediate frequency (IF) signalthat can be a manipulated by the processor 540. The receiver 530 mayalso be further operational to convert the received pulsed laser signalsinto digital representations. These digital representations mayrepresent the received pulsed laser signal or the converted IF signal.

Processor 540 is operative to generate control signals which control thereceiver 530 and the transmitter 510. These control signals may beoperable to control the pulse rate of the laser pulse and the pulsewidth of the pulse. In addition, the control signals may control thereceiver 530 such that the receiver 530 is operative to receivereflected pulsed laser signals at differing pulse rates and pulsewidths. In an exemplary embodiment, the processor generates a controlsignal such that the transmitter 510 transmits a laser pulse with aknown pulse duration. Thus, the laser is transmitted for a known amountof time for each pulse. The processor 540 further generates a controlsignal such that the receiver 530 is operative to receive a reflectedrepresentation of the laser pulse, and determines or records the pulseduration of the received laser pulse.

Once the data has been received from the transmitter 510 and/or thereceiver 530, the processor 540 determines the distance to an object inresponse to the data representing the reflected representation of thelaser. Furthermore, the processor 540 compares the duration of thetransmitted laser pulse to the duration of the corresponding receiveddata pulse. If the received laser pulse has a greater time duration thatthe transmitted laser pulse, then it can be assumed that the light pulsewas incident on an inclined surface, as a LiDAR pulse will widen uponreflection from an inclined surface due to the increased propagationtime of one portion of the light pulse. For example, at 60 m, a 0.5°beam creates λ=0.5 m distance spread at 45° surface. If g(t) is theshape of a pulse reflected from a perpendicular surface, then aninclined surface will produce the following waveform:

${s(t)} = {{\frac{1}{\Delta}{\int_{{- \Delta}/2}^{\Delta/2}{{g\left( {t - \frac{2\; x\; \tan \; \alpha}{c}} \right)}{dx}}}} = {\frac{1}{\Delta}{\int_{{- \Delta}/2}^{\Delta/2}{\sum\limits_{k}\; {{\hat{g}}_{k}{\exp \left\lbrack {2\pi \; {{jk}\left( {t - \frac{2\; x\; \tan \; \alpha}{c}} \right)}} \right\rbrack}{dx}}}}}}$

Turning now to FIG. 6, an exemplary embodiment of a proposed system ofcross segmented detection 600 in a VCSEL array 610 for improved LiDARscanning is shown. An array LiDAR 610 uses an array of lasers toilluminate a scene. The array LiDAR 610 is shown with the individualVCSEL arranged in a rectangular array. Each of the VCSEL have the sameorientation on the substrate and a dispersing lens 615 is used to spreadthe laser beams across the field of view. Switching the individuallasers creates scanning of the scene.

The VCSEL array 610 is divided into M×N blocks in order to facilitateparallel acquisition and reduction of acquisition times. In thisexemplary embodiment, the VCSEl array is divided into 3×2 segments. Eachof the corresponding VCSEL in each segment are programmed to transmitsimultaneously. Thus, in this example, the 1,1 VCSEL in each segmentstransmit simultaneously, followed by the 2,1 VCSEL in each segment, etc.Thus, all of the VCSELs in the array are transmitting at least every sixtransmission cycles. The proposed solution allows to speed up pointcloud acquisition by operating several VCSELs in parallel. Theconsidered scheme trades complication of Rx circuitry for fasteracquisition/higher frame rate. Circuitry complication is linear innumber of horizontal and vertical segments (M+N) while the speed-up isquadratic (M×N). This in contrast to TOF cameras, where for simultaneousacquisition of M×N points, M×N amplifying/sampling circuits are needed.Additional functionality of peak sorting must be implemented in the DSPdomain

One of the disadvantages to operating several VCSELs in parallel is thecreation of angular ambiguities, since a simple detector produces noinformation as regarding the source of the detected signal. To addressthis disadvantage, the proposed system employs at least two segmentedreceivers 612, 614. A first receiver 612 has M vertical segments and asecond receiver 614 with N horizontal segments. Transmit and receiveoptics ensure that pulses transmitted from block #(i,j) will be receivedin the i-th vertical segment and j-th horizontal segment. Receivedsignals in each segment will contain pulses originating from differentblocks. Pulse sorting is performed by cross-comparing digitized Rxtraces from different segments. If a pulse falls across the edge of twosegments, a partial power signal will be received by each segment. Thisratio can be used to more accurately determine the transmitting VCSEL.Thus, by determining the ratio of pulse power in each of two or morehorizontal segments and two or more vertical segments, the receiverprocessor can accurately determine the source of the pulse. Each of thesegments is equipped with an amplifier 620 and an analog to digitalconverter 622. The amplified and digitized signal from each segment isthen coupled to a processor, such as a digital signal processor.

Turning now to FIG. 7, a flowchart 700 of a method for improved LiDARscanning in accordance with an exemplary embodiment is shown. The methodis operative to receive a first signal from a first detectorrepresentative of a first light pulse 710. In this exemplary embodiment,the first detector may have a plurality of vertical segments wherein thelight pulse is dispersed across one or more of the vertical segments bya dispersing lens. The first signal may be a power signal received fromone or more of the vertical segments, or the first signal may be digitalsignal from an analog to digital converter wherein the first signal is adigital representation of the amplitude or another parameter of thelight pulse. The digital representation may further include a ratio inresponse to the light pulse detected by more than one of the verticalsegments and an identification of the segments.

The method is then operative to receive a second signal from a seconddetector representative of a second light pulse 720. In this exemplaryembodiment, the second detector may have a plurality of horizontalsegments wherein the light pulse is dispersed across one or more of thehorizontal segments by a dispersing lens. The second signal may be apower signal received from one or more of the horizontal segments, orthe second signal may be digital signal from an analog to digitalconverter wherein the second signal is a digital representation of theamplitude or another parameter of the light pulse. The digitalrepresentation may further include a ratio in response to the lightpulse detected by more than one of the horizontal segments and anidentification of the segments.

The method is then operative to determine a segment of the firstdetector and a segment of the second detector 730. A data signal isreceived from each of the first receiver and the second receiver. Thedata signal may arrive from an A/D converter dedicated to a specificsegment of a receiver, or the data signal may come from a multiplexer orthe like with an indicator of the segment of origin. The data signal mayfurther indicate more than one segment and/or a ratio of power oramplitude received at each segment.

The method is then operative to determine an origin of the light pulsein response to the segment of the first detector and the segment of thesecond detector 740. An algorithm is used to determine the origin VCSELfrom the transmitter in response to the determination of the segment inthe first detector and the segment in the second detector. For example,knowing the segment and the location of the segment relative to thediffusing lens facilitates calculation of the angle of incidence of theincoming light pulse. Alternatively, a lookup table may be used todetermine the origin VCSEL from the determined segments.

The method is then operative to generate a control data in response tothe first light pulse, the second light pulse and the determination ofthe origin 750. The control data may include the VCSEL of origin, theangle of incidence and the propagation time, or may be an obstacle mapgenerated, in part, from the determined and calculated data.

It will be appreciated that while this exemplary embodiment is describedin the context of a fully functioning computer system, those skilled inthe art will recognize that the mechanisms of the present disclosure arecapable of being distributed as a program product with one or more typesof non-transitory computer-readable signal bearing media used to storethe program and the instructions thereof and carry out the distributionthereof, such as a non-transitory computer readable medium bearing theprogram and containing computer instructions stored therein for causinga computer processor to perform and execute the program. Such a programproduct may take a variety of forms, and the present disclosure appliesequally regardless of the particular type of computer-readable signalbearing media used to carry out the distribution. Examples of signalbearing media include: recordable media such as floppy disks, harddrives, memory cards and optical disks, and transmission media such asdigital and analog communication links.

1. An apparatus comprising: a laser emitting device for emitting a laserin a first direction; a first detector having a first detection portionfor detecting a first laser intensity and a second detection portion fordetecting a second laser intensity, wherein the first detection portionand the second detection portion are oriented in a first direction, thefirst detector further operative to generate a first control signal inresponse to the detection of the first laser intensity and the secondlaser intensity; a second detector having a third detection portion fordetecting a third laser intensity and a fourth detection portion fordetecting a fourth laser intensity, wherein the third detection portionand the fourth detection portion are oriented in a second directionwherein the second direction is orthogonal to the first direction, thesecond detector further operative to generate a second control signal inresponse to the detection of the third laser intensity and the fourthlaser intensity; a first lens for distributing a first laser refectionover a subsection of the first detection portion and the seconddetection portion; a second lens for distributing a second laserrefection over a subsection of the third detection portion and thefourth detection portion; and a processor for determining a transmitterof origin in response to the first control signal and the second controlsignal.
 2. The apparatus of claim 1 wherein the first laser emittingdevice is a vertical cavity surface-emitting laser.
 3. The apparatus ofclaim 1 wherein the apparatus is part of a LIDAR system.
 4. Theapparatus of claim 1 where the first laser reflection is a reflection ofthe laser from a target within a field of view.
 5. The apparatus ofclaim 1 wherein the second detector has a fifth detection portion fordetecting a fifth laser intensity and wherein the processor can furtherdetermine the transmitter of origin by determining a ratio of the thirdlaser intensity, the fourth laser intensity and the fifth laserintensity.
 6. The apparatus of claim 1 wherein the first lens defocusesthe first laser reflection in order to distribute the first laserrefection over a greater portion of the first detection portion and thesecond detection portion.
 7. The apparatus of claim 1 wherein the laseremitting device, the first detector and the second detector have fixedorientations.
 8. A LiDAR system comprising: a transmitter fortransmitting a light pulse; a first detector having a first detectionportion and a second detection portion, the first detection portion andthe second detection portion having a first orientation; a seconddetector having a third detection portion and a fourth detectionportion, the third detection portion and the fourth detection portionhaving a second orientation, wherein the second orientation isorthogonal to the first orientation; and a processor for determining alocation of the transmitter in response to a first ratio of the lightpulse detected by the first detection portion and the second detectionportion and a second ratio of the light pulse detected by the thirddetection portion and the fourth detection portion.
 9. The LiDAR systemof claim 8 further comping a first lens for distributing the light pulseover the first detection portion and the second detection portion. 10.The LiDAR system of claim 8 wherein the transmitter is an array ofvertical cavity surface emitting lasers.
 11. The LiDAR system of claim 8wherein the processor is further operative to determine a location of anobject in response to the location of the transmitter, the first ratioand the second ratio.
 12. The LiDAR system of claim 8 wherein the seconddetector has a fifth detection portion for detecting a fifth laserintensity and wherein the processor is further operative to determinethe transmitter of origin by determining a ratio of the third laserintensity, the fourth laser intensity and the fifth laser intensity. 13.The LiDAR system of claim 8 further comprising a first lens fordefocusing the light pulse in order to distribute the light pulse over agreater portion of the first detection portion and the second detectionportion.
 14. The LiDAR system of claim 8 wherein the transmitter, thefirst detector and the second detector have fixed orientations.
 15. Avehicle sensor system comprising: a VCSEL array having a firsttransmitter for transmitting a first light pulse and a secondtransmitter for transmitting a second light pulse; a first detectorarray having a first vertical detector for receiving a first portion ofthe first light pulse and a second vertical detector for receiving afirst portion of the second light pulse, the first detector furtheroperative to generate a first control signal in response to the firstportion of the first light pulse and the first portion of the secondlight pulse; a second detector array having a first horizontal detectorfor receiving a second portion of the first light pulse and a secondhorizontal detector for receiving a second portion of the second lightpulse, the second detector further operative to generate a secondcontrol signal in response to the second portion of the first lightpulse and the second portion of the second light pulse; and a processorfor determining a location of an object in response to the first controlsignal and the second control signal.
 16. The vehicle sensor system ofclaim 15 wherein the first light pulse and the second light pulse aretransmitted simultaneously.
 17. The vehicle sensor system of claim 15where the object is within the field of view of the VCSEL array, thefirst detector and the second detector.
 18. The vehicle sensor system ofclaim 15 further comprising a lens for dispersing the first light pulseand the second light pulse over the first detector and the seconddetector.
 19. The vehicle sensor system of claim 15 further comprising afirst lens for dispersing the first light pulse over the first detectorand a second lens for dispersing the first light pulse over the seconddetector.
 20. The vehicle sensor system of claim 15 further comprising acontroller for controlling the vehicle in response to the determinedlocation of the object.